The invention relates generally to the field of digital communications, and more particularly to systems and methods for queuing data being transferred between devices such as switching nodes in a digital data network.
In digital communications systems, data can be transferred from an input or transmitting device to an output or receiving device. In such systems, it often occurs that data cannot immediately be transferred from the input device to the output device for some reason such as congestion or high data traffic at the input device and/or the output device. To solve this problem, such systems implement a queuing function which can buffer or queue data at the input and/or output devices. Input-queued data can be temporarily stored until it can be transferred to the output device. Output-queued data, which has been transferred from the input device, can be temporarily stored until it is forwarded to its eventual destination by the output device. One application for data queuing is in digital networks.
Digital networks have been developed to facilitate the transfer of information including data and programs among digital computer systems and numerous other types of devices. A variety of types of networks have been developed and implemented using diverse information transfer methodologies. In modem networks, information is transferred through a mesh of switching nodes which are interconnected by communication links in a variety of patterns. The mesh interconnected pattern can allow for a number of paths to be available through the network from each computer system or other device.
Information transferred from a source device to a destination device is generally transferred in the form of fixed-length or variable-length data packets, each of which is in general received by a switching node over a communication link and transmitted over another communication link to facilitate transfer of the packet to the destination device or to another switching node along a path to the destination device. Each packet typically includes address information including a source address that identifies the device that generated the packet and a destination address that identifies the particular device or devices which are to receive the packet.
Typically, a switching node includes one or more input ports, each of which is connected to a communication link on the network to receive data packets, and one or more output ports, each of which is also connected to a communication link on the network to transmit packets. Each node typically also includes a switching fabric that couples data packets from the input ports to the outport ports for transmission.
When an input port receives a packet from the network, it typically buffers the packet. It uses the destination address of the packet to identify the particular output port that is to transmit the packet onto the network and then transfers the packet to the identified output port through the switching fabric. When the output port receives the packet, it typically buffers the packet in a queue where it awaits transmission onto the network over the appropriate communication link. While buffering and scheduling by the output port can provide for efficient packet transmission by the output port, since the output port can be kept continually busy, several problems can arise with conventional output port buffering. Generally, each output port effectively provides one queue for each input port, in which case the total number of queues provided by the switching node will be on the order of N2, where N is the number of input ports, which, in turn, corresponds to the number of output ports, if, as is typical, each communication link provides for bidirectional transmission of packets. Thus, as N increases, the number of queues maintained by the output ports generally increases approximately quadratically.
Queues are generally expensive pieces of hardware, and, in general, the longer the queue, the more expensive it is to implement. Switching nodes in modern networks must accommodate large amounts of data which are transferred over network links at increasingly high rates. Communication links and the switching nodes to which they are connected must accommodate large bursts of data which occur in very short amounts of time, i.e., at very high rates. The average data rates that must be accommodated can also be very high. With these increased high data rates comes an increase in the required queuing capacities of the switching nodes, which results in the requirement that larger queues be implemented. Thus, the development of larger and faster systems results in the need for increasing numbers of longer queues, which in conventional queuing approaches can in turn result in increased cost.
The present invention is directed to an apparatus and method for queuing data. The invention can be employed in systems such as switching nodes on a digital network to queue data packets that are being transferred over the network or other types of data that are being transferred within the switching node itself. The queuing system of the invention is used to queue data that is being transferred from an input device, such as an input port module in a network switching node, to an output device, such as an output port module in a network switching node. The queuing apparatus of the invention utilizes a plurality of input ports in the input device that receive data to be transferred to the output device, such as from a communication link on a digital network. Each input port is adapted to receive the data at a data rate that is associated with the input port. The device includes a plurality of short queues for storing data received at the plurality of input ports. A long queue having more data storage capability than each of the short queues receives and stores data from the plurality of short queues and forwards the stored data toward the output device. A control circuit controls the transfer of data from the short queues to the long queues. The data are transferred at a second data rate higher than at least one of the data rates associated with the short queues such that a data threshold associated with at least one of the short queues is not exceeded. That is, in one embodiment, the short queues are emptied at a rate that is fast enough the prevent the at least one short queue from becoming full.
In general, each of the input ports can be associated with and operate at a different data rate. In one embodiment, the data rate associated with the input ports is selected to be the maximum data rate of all of the data rates associated with all of the input ports. In this case, because the control circuit transfers data from the short queues to the long queues at a rate faster than the selected rate, it is insured that none of the input queues will store data beyond their thresholds. For example, none of the queues will become full.
In one embodiment, the control circuit is associated with the output device for transferring data from the associated short queues to the associated long queue. The output device can include multiple output ports, in which case, the system would include multiple control circuits, each associated with a single output port.
The control circuit can include a selection circuit which selects the short queues to enable each short queue to transfer data to the long queue. The short queues can be selected in turn in a round-robin fashion, with each short queue being selected at a rate faster than the data rate associated with the input communication link. In one particular embodiment, during the round-robin selection process, before each queue is accessed, the control circuit determines whether each queue has data to be transferred. If no data is to be transferred, the queue can be skipped. This results in saving the time that would have been allocated for retrieval of data from the particular short queue. In one embodiment, the selection circuit includes a multiplexer for selectively passing data from selected short queues to the long queue. The multiplexer is activated to select individual short queues in turn as desired.
Each of the short queues can be associated with an input port. Therefore, the output device is associated with as many short queues as there are input ports associated with the output device. In the common case in which there are as many output ports in the output device, e.g., N output ports, as there are input ports in the input device, e.g., N input ports, N2 short queues would be implemented.
The queue length required for a particular input port is a function of the allowed data rate for the associated communication link as well as the rate at which the queue is emptied. In general, the faster the rate at which the queue is emptied, the shorter the queue needs to be. In one embodiment, because the queues are emptied at a rate faster than the data rate of the communication link, the short queues are shorter than the queue that would be required in a system in which the queues were emptied at a rate equal to or less than the data rate of the associated communication link. These smaller short queues can be implemented more efficiently and less expensively than the queues that would be required in a conventional system.
The queuing apparatus of the invention can also include an additional queue for each output port. This xe2x80x9cthirdxe2x80x9d queue can receive data from the long queue. The data stored in the third queue can then be retrieved for transfer to the output device. The third queue can receive data from the long queue at a rate that is slower than the rate at which the long queue receives data from the short queues. The rate can be slower because of the smoothing effect that the long queue has on the short queues. That is, the long queue, be periodically draining the short queues and storing their data, smooths or averages any large spikes in the rate at which data is received at the input device, i.e., bursts. In one embodiment, such as where the queuing apparatus of the invention is implemented in a network switching node in which the output device includes one or more output ports connected to one or more communication links, the rate at which the third queue is emptied can be adjusted to accommodate a data rate associated with the output device, e.g., a data rate at which a communication link connected to an output port operates.
As described above, the queuing apparatus and method of the invention be implemented in a switching node on a network. The switching node to which the queuing apparatus and method of the invention can be applied can include a plurality of input ports that receive data from the network. Each input port can receive data over a respective associated communication link of the network at a data rate associated with the respective communication link. The node also includes at least one output port that transfers data onto the network over a communication link. The output port can receive data from a plurality of associated input ports of the switching node. Associated with the at least one output port is a plurality of short queues. The short queues store data received at the input ports. A long queue, that is, a queue having more data storage capability than each of the short queues, is associated with the output port. It receives and stores data from the plurality of input ports that are associated with the output port. That is, data from the associated short queues are transferred to the long queue. A control circuit is implemented to control transfer of the data from the short queues to the long queue. The control circuit removes data from the short queues and transfers it to the long queue at a rate faster than at least one of the data rates associated with the input ports, such that a threshold associated with at least one of the short queues is not exceeded. For example, a threshold may be selected for each queue to indicate when it is fill. This structure allows the required queuing capacity to be implemented in multiple short queues and one relatively longer queue, resulting in increased queuing capacity and reduced overall expense.
Each of the communication links associated with input ports of the switching node can, in general, operate at a different rate. This can be attributed to the fact that the links can be used by a variety of users which transfer data at different rates and/or are constrained to different data rates in accordance with their individual contractual usage requirements. In one embodiment, the data rate associated with the input ports is selected to be the maximum data rate at all of the associated communication links. The result is that when the control circuit removes data from the short queues at a faster rate, none of the queues will exceed their thresholds.
In one embodiment, the control circuit is associated with the output port for transferring data from the associated short queues to the associated long queue. The switching node can include multiple output ports; therefore, the node can include multiple control circuits, each of which is associated with the output port. The control circuit can include a selection circuit which selects the short queues to enable each short queue to transfer data to the long queue. The short queues can be selected in turn in a round-robin fashion, with each queue being selected at a rate faster than the at least one data rate associated with the input communication links, such that it is ensured that the threshold data levels in the short queues are not exceeded. In one particular embodiment, the selection circuit includes a multiplexer for selectively passing data from selected short queues to the long queue. The multiplexer is activated to select individual short queues in turn as desired.
The apparatus and method of the invention used to efficiently implement data queuing is applicable in various networks in which data queuing is used. For example, the invention can be implemented in a switching node of the type described in copending U.S. patent application Ser. No. 09/108,771, filed on Jul. 2, 1998, entitled xe2x80x9cSystem and Method for Switching Packets in a Network,xe2x80x9d by Schwartz, et al, and assigned to the same assignee as the present application. The contents of that application are incorporated herein in their entirety by reference.
As mentioned above, the data queuing apparatus and method of the invention provide a more efficient and less expensive queuing implementation than prior approaches. Because multiple short queues are used, less hardware is required to implement data queuing. As a result, less expensive hardware can be used, resulting in a less expensive overall system.